Multi-monitor computer systems, i.e., a computer system with more than one display/monitor, are becoming more and more prevalent. Their prevalence is due, at least in large part, to the fact that (1) the typical graphic display subsystem provides support for more than one monitor and (2) the cost for a second monitor is relatively small. Another factor is that a multi-monitor computer system offers a computer user more area upon which information and work may be displayed. With a greater display area, the computer user spends less time cycling through overlapping windows, frequently referred to as “thrashing,” to find information that may lie hidden under the overlapping windows.
FIG. 1A is a pictorial diagram illustrating an exemplary multi-monitor computer system 100. As shown in FIG. 1A, a computer 102 is attached to two monitors, monitor 104 and monitor 106. However, as most multi-monitor computer users will readily appreciate, and as shown in FIG. 1A, the monitors in a multi-monitor computer system are frequently dissimilar, both in physical size as well as other aspects described below. This is especially true when the multi-monitor computer system is based on a laptop, a tablet computer, or a personal digital assistant (PDA), all with integrated display units.
In addition to differences in physical size, monitors in a multi-monitor computer system may also be dissimilar with regard to screen resolution, i.e., the number of pixels that are displayed on the displayable area of a monitor. As an example, with regard to the exemplary multi-monitor computer system 100, monitor 104 may have a screen resolution of 1024×768 pixels, whereas monitor 106 may have a screen resolution of 1280×1024 pixels. Those skilled in the art will recognize that these screen resolution values, such as 1024×768 pixels, refer to the number of pixels displayed in each row of pixels (e.g., 1024 pixels per row) by the number of rows of pixels (e.g., 768 rows of pixels). Thus, as appreciated by those skilled in the art, a monitor with a greater screen resolution can display more raw data than a monitor with a lower screen resolution simply because the monitor with the greater screen resolution has more pixels available to display the data.
Monitors also frequently differ with respect to pixel resolution, i.e., the size of each pixel displayed by the monitor. All other factors being equal, monitors with a lower pixel resolution display the same information in less physical area than monitors with a lower pixel resolution. Thus, the monitor with a greater screen resolution is not necessarily physically larger than the monitor with a lesser screen resolution. For example, with reference to FIG. 1A, while monitor 106 is physically larger than monitor 104, both monitors could have the same screen resolution, with monitor 104 displaying its information in a smaller display area due to a smaller pixel resolution. Furthermore, due to a smaller pixel resolution, monitor 104 could have a greater screen resolution than monitor 106.
While multi-monitor computer systems are generally very desirable, due to the differences between monitors, as well as the physical separation and physical alignment of multiple monitors, numerous problems arise with regard to displaying information across the multiple monitors. In particular, some of these problems arise with regard to the visual continuity of a pointer as the pointer is moved across monitor boundaries from a source location to a target location. While also referred to as a “cursor,” i.e., the visual image or icon representing a current location on the multi-monitor display area, for purposes of the present invention will use the term “pointer.”
For purposes of the present discussion, moving the pointer from a source location to a target location will be generally referred to as “target acquisition.” Additionally, the present discussion will refer to a mouse as the input/control device by which the pointer is moved. However, reference to a mouse is for simplicity in description only, and should not be construed as limiting upon the present invention. Those skilled in the art will appreciate that the pointer may be controlled/moved through the use of any number of input/control devices, including, but not limited to, a mouse, touchpad, joystick, pointer keys, and the like.
In conjunction with a mouse, target acquisition relies heavily upon hand-eye coordination based on the spatial arrangement of displayed items. In other words, target acquisition includes both visual and spatial aspects. In this paradigm, the computer user has a legitimate expectation that mouse movements will directly correspond to pointer movements. When mouse movements correspond to pointer movements, target acquisition is facilitated. Conversely, when pointer movements fail to directly correspond to mouse movements, target acquisition is impeded.
On computer systems that include just a primary monitor, the mouse movements almost always directly correspond to pointer movements. Unfortunately, on multi-monitor computer systems, the visual/spatial correlation between mouse movements and pointer movements is almost always disrupted when the pointer crosses monitor boundaries. For example, with regard to FIG. 1A, assume that a user wishes to move the pointer 112 from the source 108 on monitor 104 to the target 110 on monitor 106. As illustrated in FIG. 1A, visually, the source 108 and the target 110 are horizontally aligned, as indicated by line 114. Thus, a user would acquire the target 110 on monitor 106 by moving a mouse (not shown) to the right on a horizontal line with the expectation that the pointer 112 will move in a corresponding toward the target 110.
However, on current multi-monitor computer systems, due to any number of conditions, including, but not limited to, the physical separation of monitors 104 and 106, the physical alignment of the monitors, the internal arrangement of the display surface, the screen resolution of the monitors, and the display resolution of the monitors, the user's expectations are not met.
FIG. 1B, a pictorial diagram illustrating the multi-monitor computer system 100 of FIG. 1A, illustrates the displayed path 116 of the pointer 112 as the user moves the mouse in a horizontal line to its right. Line 114 in FIG. 1B illustrates how the pointer actually “tracks,” i.e., is displayed, with the movements to the mouse. As can be seen in FIG. 1B, as the pointer 112 tracks across monitor boundaries from source 108 towards target 110, the cursor jumps, or “warps,” across from monitor boundary 116 to monitor boundary 118, the latter being a location not on the horizontal line between the source 108 and target 110. Additionally, while the horizontal mouse movement was small, visually, the pointer's 112 horizontal movement was substantial, i.e., spanning the frames and physical separation of monitors 104 and 106. Clearly, this visual discontinuity is extremely disruptive to target acquisition. Psychophysical studies have demonstrated that the bulk of a pointing movement takes place in a rapid, ballistic mode that is preplanned, without reliance on visual feedback in the early phase of target acquisition. Hence a discontinuity in the pointer trajectory as it crosses a boundary between monitors is disruptive to human performance, and is very difficult for a user to compensate for.
It should be noted, however, that while visually, the pointer 112 warped to an unexpected location, to the system, the pointer 112 tracked along the horizontal line of the mouse without any jumps or warps. As those skilled in the art will appreciate, while multi-monitor computer systems are generally aware that multiple monitors are connected to the system, they are almost universally unaware of physical alignment, separation, and screen resolution issues that affect the visual display of information across the multiple monitors. To the current multi-monitor computer system, conceptually, the internal representation of these monitors is a single, contiguous display surface encompassing the display areas of all monitors. For example, FIG. 1C is a block diagram of the internal display surface 150 of the exemplary multi-monitor computer system 100 of FIGS. 1A and 1B. As shown in FIG. 1C, the display surface includes display areas 120 and 122 corresponding to monitors 104 and 106 respectively.
With regard to the internal display surface 150, a user may be able to specify the arrangement of the multiple display areas to each other, or alternatively, specify the arrangement of one display area to a primary display area. However, the display areas are viewed as contiguous by the computer system 100. However, they do not necessarily reflect the physical arrangement of the corresponding monitors, as a user is free to rearrange the monitors without informing the system 100. Being contiguous, the display areas clearly cannot account for the physical separation between them, nor do they necessarily account for the actual physical alignment of the monitors. As shown in FIG. 1C, in the display surface 150, display areas 120 and 122 are aligned along their bottom. However, as can be seen in FIG. 1B, the bottoms of the displayable areas of monitors 104 and 106 are not aligned.
In addition to physical conditions that exist between monitors, while some current multi-monitor computer systems are aware of the screen resolutions of its monitors, current multi-monitor computer systems are unaware of the pixel resolutions of those monitors. To the multi-monitor computer system 100, each pixel is the same size, irrespective of the corresponding display area/monitor. Unfortunately, as discussed above, monitors frequently differ in pixel resolution and pixel resolution can have a substantial impact on the visual appearance of displayed information, including a pointer 112. Thus, returning to the example of FIGS. 1A and 1B, as the mouse is moved in a horizontal line, visually towards the target 110, the pointer 112 tracks along a logical horizontal line 124 on the display surface 150. Additionally, while source 108 and target 110 visually appear to be horizontally aligned in FIGS. 1A and 1B, as shown in FIG. 1C, to the computer system 100, they are not horizontally aligned.
While FIGS. 1A-1C illustrate some of the display issues, or anomalies, that arise when displaying information across monitors on a multi-monitor computer system, there are others that create just as much confusion and frustration for a user. FIGS. 2A and 2B are pictorial diagrams of the exemplary system 100 for illustrating some of these additional anomalies. In the example illustrated in FIG. 2A, the source 108 is now found on monitor 106 and the target 110 on monitor 104. Visually, both source 108 and target 110 are horizontally aligned. Thus, to the computer user, it would appear that, in order to acquire the target 110, the user must move the pointer 112 on a direct horizontal line to the left, as indicated by line 204
Unfortunately, as illustrated in FIG. 2B, as the user encounters the edge of the display area of monitor 106, the pointer 112 is stopped from crossing over to monitor 104 by an unseen barrier 202. This unseen barrier exists due to various differences and conditions between monitors 104 and 106 as described above. To better explain, FIG. 2C is a block diagram illustrating the exemplary internal display surface 150 of the multi-monitor computer system 100. Source 108 and target 110 are shown at the locations where the computer understands them to be. Similar to their internal arrangement discussed above in regard to FIGS. 1A-1C, in the display surface 150 of FIG. 2C, source 108 and target 110 are not horizontally aligned.
As discussed above, display areas 120 and 122 are aligned along their bottom edge. Because display area 122 is larger than display area 120 (meaning that for this example, the screen resolution for monitor 106 is greater than the screen resolution for monitor 104), a top segment of display area 122 is not contiguous with any portion of display area 120. Furthermore, as can be seen in FIG. 2C, the computer system 100 believes that source 108 resides in this top portion. Accordingly, as the pointer 112 is tracked to its left on a horizontal line, the edge of the display surface 150 is encountered and further travel in that direction is prohibited, hence the invisible, apparent barrier 202.
FIG. 3A is a pictorial diagram illustrating an exemplary multi-monitor computer system 300 for further illustrating the pointer display issues described above. As shown in FIG. 3A, this exemplary multi-monitor computer system 300 includes three monitors/display devices, including the tablet computer's 302 integrated display, as well as monitors 304 and 306. For purposes of the present discussion, with regard to the multi-monitor computer system 300, it will be assumed that monitors 304 and 306 are the same type of monitor having the same screen resolution, 1280×1024 pixels. Furthermore, the tablet computer's 302 integrated display, according to its current orientation, has a screen resolution of 768×1024 pixels.
As illustrated in FIG. 3A, the source 108 and pointer 112 are on the tablet computer's 302 integrated display and the target is on monitor 306. As illustrated by line 304, visually, the user wishes to move the pointer 112 from source 108 to target 110 in a direct line. However, as the pointer 112 attempts to cross out of the tablet computer's 302 display area towards the target 110, the pointer is again stopped by an invisible barrier, i.e., is not actually displayed to the user, as indicated by a barrier 308.
FIG. 3B illustrates the exemplary display surface 350 of the exemplary multi-monitor system 300. As illustrated on the exemplary display surface 350, as the pointer 112 tracks from the source 108 on a direct line to the target 110, the user encounters the edge of the display area 318 corresponding to the tablet computer's 302 display. According to the arrangement of the internal display surface 350, there is no display area immediately to the right of display area 318. Thus, the pointer 112 cannot continue in its current direction toward target 110, hence the apparent, invisible barrier 308. For the user, in order to acquire the target 110, the user must move the pointer 112 up into the display area 314 corresponding to monitor 304, and then to the target 110 on display area 316 corresponding to monitor 306.
As clearly illustrated in the above examples, differences in physical size, screen resolution, pixel resolution, physical and internal alignments, and physical separation all affect the visual display continuity of the pointer 112 as the user moves a pointer from a source 108 to a target 110 across multiple monitors. Thus, in order to provide visual continuity as displaying the pointer, a computer system must first be able to determine those differences. A novel invention to determine these differences and dissimilarities between monitors in a multi-monitor computer system has been set forth in co-pending, and commonly assigned, U.S. patent application Ser. No. 10/884,537, filed Jul. 2, 2004, entitled “System and Method for Determining Display Differences Between Monitors on Multi-Monitor Computer Systems,” which is incorporated herein by reference (hereafter referred to as the “incorporated reference”). As described in the incorporated reference, a computer system displays user-actionable information across multiple monitors such that based on minimal user interactions, the computer system can determine the relative differences in screen resolutions, pixel resolutions, physical alignment, physical separation, and rotation of two monitors in a multi-monitor computer system. This determined information can then be advantageously used by a software application to display information on the multi-monitor computer system. The interaction between the computer system and the user to determine the relative differences in screen resolutions, pixel resolutions, physical alignment, physical separation, and rotation of monitors in a multi-monitor computer system will be referred to hereafter as the “calibration process,” and the determined information will be referred to as the “calibration information.”
Those skilled in the art will recognize that alternative calibration processes may be employed. For example, for displays that include a touchscreen or pen input surface, sweeping one's finger or a pen across the two displays can be used to compute the physical alignment and rotation of the two displays. As an alternative, a calibration process could be completed by drawing a line with a mouse on one display, and then drawing a matching line with a mouse on another display. In some cases, calibration might even be completely automated by using range sensors, such as those used in cameras to sense the distance to a subject. Monitors equipped with such sensors mounted to the bezel could sense the distance to and alignment of displays incorporating proximity sensors. The system of the present invention could use such automatically generated sensor information in lieu of, or in conjunction with, one or more explicit user interactions to indicate physical alignment, physical separation, and rotation between monitors.
While the calibration information may be determined and advantageously used by software applications, unfortunately, in current multi-monitor computer systems, the pointer 112 can not. As indicated above, the pointer (as well as its display) is intimately tied to the computer system's internal display surface, such as display surfaces 120 or 350. Thus, even when software applications compensate for display differences between monitors, the visual display of the pointer 112 is tied to the computer system's display surface, and is therefore unable to take advantage of the determined calibration information.
In light of the above-described issues, what is needed is a system and method that displays the pointer, in a visually correct and/or consistent fashion, in a multi-monitor computer system. The present invention addresses these and other issues found in the prior art.